Fast Algorithms for Structured Sparsity

Sparsity has become an important tool in many mathematical sciences such as statistics, machine learning, and signal processing. While sparsity is a good model for data in many applications, data often has additional structure that goes beyond the notion of “standard” sparsity. In many cases, we can represent this additional information in a structured sparsity model. Recent research has shown that structured sparsity can improve the sample complexity in several applications such as compressive sensing and sparse linear regression. However, these improvements come at a computational cost, as the data needs to be “fitted” so it satisfies the constraints specified by the sparsity model. In this survey, we introduce the concept of structured sparsity, explain the relevant algorithmic challenges, and briefly describe the best known algorithms for two sparsity models. On the way, we demonstrate that structured sparsity models are inherently combinatorial structures, and employing structured sparsity often leads to interesting algorithmic problems with strong connections to combinatorial optimization and discrete algorithms. We also state several algorithmic open problems related to structured sparsity.